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ABSTRACT 

This report describes a staff training and 
performance monitoring system for elementary school teachers, teacher 
aides, and parent aides participating in the Behavior Analysis Follow 
Through (B«A.) compensatory education program* The B.A* model 
involves the use of additional staff, individualized instruction, 
programmed curriculum materials, and a token motivation system* The 
steps involved in the training*monitoring sequence are: (1) providing 
a rationale for desired performance, (2) modeling of the described 
behavior, and (3) providing the trainee with adequate verbal feedback 
and praise. Demonstration and training classes, as well a^s training 
manuals are provided for the training period. Trainers establish a 
schedule of classroom monitoring to assure continuing maintenance of 
program objectives* Data are presented which indicate that: (1) the 
staff training procedures are successful, (2) student performance 
convaries with relative adherance to the classroom model, (3) the 
instructional model is replicated across project sites and grade 
levels, and («) the teacher training and support system are 
acceptable to school personnel* (Author/BItT) 
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Abstract 

To have a wide-spread Impact, an education program must use 
standard and economical procedures for staff training and performance 
monitoring. Accordingly, the Behavior Analysis Follow Through pro- 
gram established a training/monitoring system to assure quality 
performance In each elementary school classroom using the Behavior 
Analysis model. The steps Involved in staff training and subsequent 
support of quality performance are described. Data are presented 
indicating that the staff training procedures are successful, that 
student performance covaries with relative adherence to the instruc- 
tional model, that the instructional model is replicated across 
project sites and grade levels, and that the teacher training and - 
support system are acceptable to school personnel. 
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Monitoring Staff Performance: 
A Plan for Quality Control in Project Follow Through 

Behavior Analysis Follow Through (B.A,), a comprehensive class- 
room instructional model sp'^n**ored by the Department of Human Develop- 
ment at the University of Kansas, has been successful at improving 
the academic performance of thousands of poor children who, on an 
actuarial basis, were expected to fail in school* In hundreds of 
Behavior Analysis classrooms scattered throughout the nation in a 
variety of communities and school settings (see Table 1), children 
in kindergarten through third grade are successfully meeting grade* 
level norms (Abt & Associates, 1973; Bushell, in press). 



Insert Table 1 about here 



A social intervention as massive as Behavior Analysis faces 
unprecedented problems in implementing and maintaining the pro- 
gram model in all of its remote sites. In such large-scale appli- 
cations, standard and economical procedures for assuring quality 
performance in each site become essential. This paper describes 
the basic staff training and performance monitoring system used 
in the Behavior Analysis program to assure adequate implementation 
in all classrooms. Data are presented which indicate that the 
staff training procedures successfully modified teacher behavior, 
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that student performance covaried with the relative adherence to 
the classroom model, that the instructional model was replicated 
across project sites, and that the teacher training methods were 
acceptable to school personnel. 

The basic B.A. model includes an expanded classroom staffing 
pattern, individualized instruction, programmed curriculum materials, 
and a token motivation system* (Also see Bushell, 1973; Bushell & 
Ramp, 1974*) A careful specification of all of the critical elements 
of the model provided the foundation for building the teacher train- 
ing and perforniance monitoring system* 

Once the program model had been specified, the first step in 
building the training/monitoring system was to find a way of pro- 
viding immediate feedback on the academic performance of each child 
each week* The availability of high-speed computerized data trans- 
mission and processing made it possible for each teacher to receive 
rapid feedback on each child's progress toward individually prescribed 
curriculum targets* With this basic monitoring system working, 
the next step was to devise an effective means of training people 
to use the teaching methods, and once trained, of assuring their 
continued use with minimum supervision* The problem demanded a 
procedure that would assure the replication of the program model 
in all B,A. sites with minimum supervision from the University of 
Kansas. 
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The basic package for training the classroom staff (teachers^ 
teacher aides» and parent aides) begins with several lists of teach- 
ing performance and outcome criteria^ which specify critical instruc- 
tional methods as well as form the basis for public recognition of 
good teaching performance. 

One set of performance criteria is to be met in the classroom 
instructional situation. Figure 1 lists the Instructional Teaching 
Criteria. Observations made by a staff trainer or parent trainer 



Insert Figure 1 ab*. t here 



must yield data that support a **yes" answer to each of the items 
shown. Teaching behaviors as well as the child's on-task and 
accuracy are checked. 

A second set of performance criteria is used to evaluate the 
operation of the classroom motivation system. Figure 2 shows the 



Insert Figure 2 about here 



Exchange Teaching Criteria. If observations support **yes** answers 
to each it^ on this list» the trainer can be assured that the token 
motivation system is functioning properly. 

The final performance criteria requires that* fbr a period of 
four weeks > 80SS of the children in the teacher's own instructional 
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groups be reported as having met their Individualized progress 
targets, as specified by the computer feedback system. 

If two observations, taken at least four weeks apart, both 
show that the teacher or aide has met all of the criteria and is 
maintaining at least 80SS of her children "on target", this per- 
formance is acknowledged with a Behavior Analysis specialist cer- 
tificate and a letter of commendation. Copies of the letter go to 
the principal and to the local Board of Education office for place- 
ment in the teacher's personnel fil^. This certification procedure 
is repeated yearly* Thus, progress toward B,A* certification might 
be used as an indication of the degree of adherence to the model 
In each of the B,A, sites. 

To train the classroom staff to meet these criteria, each 
Behavior Analysis project site employs its own trainers. Most 
projects also operate demonstration and training classes for pro- 
viding intensive individualized practicum experience for new staff. 
In addition, training manuals written specifically for the B*A* 
trainers describe training and observation procedures in detail 
(Nelson, Saudargas, & Jackson, 1974; Jackson, Minnis-Hazel, & 
Saudargas, 1974). 

The staff training manuals specify a general three-step 
training process used both in the training of new teachers and the 
monitoring of and follow up with experienced teachers. These steps 
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are; (a) giving a rationale for the desired performance, describ- 
ing those behaviors, and setting a criterion (such as always using 
words of praise that describe the behavior being praised); (b) 
modeling the desired behavior(s) and then having the trainee try it/ 
them; and (c) providing the trainee with graphic feedback, verbal 
feedback and praise. The effectiveness of these procedures is 
supported in other research reports (e.g., Holt, Kolb, Bushell, & 
Jackson, 1974; Sabbert, Holt, & Jackson, 1975; Nelson, Jackson, 
Hughes, & Jenkins, 1975; Cooper, Thomson, & Baer, 1974; Clark, 
Macrea, Ida & Smith, 1975). For example, the results of the Holt 
et al.(1974) study showed that a training procedure consisting of 
instructions, modeling, and graphic and verbal feedback increased 
six trainees' rate of teacher contacts to on-task children (Fig* 3), 
and the percentage of tokens delivered along with a descriptive 
praise statement (Fig. 4) in a preschool classroom which was using 
the B.A. instructional model. 



Insert Figures 3 and 4 about here 



After initial training has been completed, the trainer will 
establish a regular schedule of monitoring observations* Itmiediately 
following each classroom observation, praise and specific feedback 
are given to the teacher. Later, the trainer will record the re- 
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suits of the observation on summary sheets in a notebook* providing 
the trainer with a continuous picture and permanent record of the 
performance of every teacher. Thus* these regular performance checks 
assure that each teacher and aide continue to receive the feedback 
and assistance that began with the B,A, certification procedures. 

The final step in the training/monitoring sequence is the 
regular evaluation provided by an external consultant* who is a 
professional behavior analyst, called the District Advisor (D.Aj, 
The D,A, is the representative of the University of Kansas Support 
and Development Center for Project Follow Through* the originator 
of the B.A, model. The D,A, uses the telephone, the mails, and 
monthly visits to a site to monitor the performance of the children 
in each classroom and to insure that the local project staff are 
receiving appropriate in-service support and technical training. 

While the training package has been shown to be effective in 
modifying teacher behavior, a more important demonstration, and a 
more difficult one* is to show the effects of the training/monitoring 
program on the children. The following four figures present an 
internal evaluation within the B,A, program that suggests what 
these effects are. 



Insert Figure 5 about here 
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Figure 5 shows the percentage of first* second* and third graders 
in one B*A* project site who were meeting their individualized 
curriculum progress targets during three periods of the school year* 
The cross-hatched bars indicate the performance of children in 
seven classrooms whose teachers met the B*A* certification criteria* 
and the open bars represent the performance of children in the re- 
maining eight classrooms where the teachers did not meet the certi- 
fication criteria* The graph shows that not only was the performance 
of the children with certified teachers better throughout the year* 
but also that this performance was maintained following the use 
of the teacher certification procedures* 

The relevance of the B*A* teaching criteria to student perfor- 
mance is further supported by indications that children taught by 
teachers meeting all of the B*A. criteria perform better on yearly 
achievement tests* For example* Figure 6 shows that in one B*A* 



Insert Figure 6 about here 



site the mean grade level score for the reading'portion of the 
Wide Range Achievement Test was four months higher for first graders 
with B*A* certified teachers than the mean score for first graders 
in this site with non-certified teachers* even though both groups 
were above the expected grade level performance* In another site* 
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where another certified vs. non-certified performance comparison 
was possible, second graders performed an average of eight months 



Insert Figure 7 about here 



better when instructed by teachers meeting all of the B.A* certi 
fication criteria (see Fig* 7), and third graders with certified 
teachers were nine months ahead of children with non-certified 
teachers (see Fig* 8)* 



Insert Figure 8 about here 



Given that the quality control (B*A* certification) criteria 
seem to be related to student performance! it is important to ask 
how successfully the program package has been implemented each 
time it's applied* To this end, independent evaluators from the 
Stanford Research Institute (SRI) examined adherence to specific 
program objectives in five of the twelve Behavior Analysis project 
sites (Stallings & Kaskowitz, 1974). Using an interval measure- 
ment scalei 35 classrooms were observed in all. The resulting 
datai surmarized as a ranking from 1 to 5 where 5 indicated the 
most adherence to the model, provide measures of how well the B.A. 
model procedures were being implemented in the classrooms observed 
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by SRK {Although a complete description of the observation and 
reliability procedures used is not possible here, they are reported 
in detail in the SRI report referenced above,) Thus, for example, 
on the SRI measure of adult feedback and praise given for individual 
child academic responses {Table 2), 17 (94%) of the 18 first grade 
classes and 12 (71%) of the 17 third grade classes in the sample 
received the highest ranking on this particular variable. 



Insert Table 2 about here 



Similar effects were reported by SRI for the percent of time in- 
struction takes place in small ungraded groups (Table 3), the 
percent of time spent in reading instruction (Table 4), and the 
degree of one-to-one adult-child interacting found in B,A, class- 
rooms (Table 5) . 



Insert Tables 3, 4, and 5 about here 



Although convincing demonstrations of "effect" are certainly 
crucial to the survival of a compensatory education program, equally 
important to its survival is the social acceptability of the methods 
that were used in achieving the effects. Responses given on yearly 
satisfaction surveys conducted by B,A. appear to indicate that, 
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by and large» the training methods are acceptable to those with 
whom they are used. The bar graph in Figure 9 indicates that, 
on a scale of 1 - 7, the mean rating given to the training program 
by all B.A. teachers and aides in Spring, 1975 was 6.0, or 



Insert Figure 9 about here 



"satisfied.** Also rated was the satisfaction of school personnel 
with the services provided by the District Advisor or other agents 
from the U;tiversity of Kansas Support and Development Center for 
Project Follow Through, Although these ratings are somewhat lower. 
Figure 10 indicates that the groups sampled generally were satisfied 



Insert Figure 10 about here 



with these support services. In yet another portion of the satis- 
faction survey, all program consumers (with the exception of non- 
FoUow Through teachers) appear to be happy with the Behavior 
Analysis classroom methods. Figure 11 reports these data. 



Insert Figure 11 about here 



The importance and relative impact of a social intervention 
strategy might best be evaluated along four dimensions; the degree 
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of success in staff training, related improvements in childrens^ 
performance, successful replication of the prdgram model in a 
variety of settings^ and adequate client satisfaction with the 
program. The results presented here suggest that the Behavior 
Analysis training/monitoring system successfully trains teacher 
behaviors, that student performance covaries with relative adher- 
ence to the instructional model, that classroom teaching procedures 
can be replicated across sites, and that the training and class- 
room procedures are generally acceptable to program consumers. 
Current efforts are directed towards a finer examination of the 
components of the training and monitoring system with the goal 
of determining the procedures essential to program maintenance. 
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Table 1 



BEHAVIOR ANALYSIS 
Follow Through Projects In Operation 
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896 
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Program initiated in the fall of 1963, All other programe initiated in the 
fall of 1969, 
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Tdble 2 

ADULT FEEDBACK TO CHILD RESPONSE TO ADULT ACADEMIC COMMANDS, REQUESTS, 
OR DIRECT QUESTIONS (Variable 412a)--UNIVERSITY OF KANSAS 



First Grade Classrooms Third Grade Classrooms 
with Implementation with Implementation 
Scores of Scores of 



Sites 1 


2 3 4 


5 


1 2 


3 


4 5 


NYC P.S. 77X 
Philadelphia VI, Pa. 
Portagevllle, Mo. 
Kansas City, Mo. 
Louisville^ Ky. 


1 


2 
3 
4 
4 
4 


1 


1 


1 1 

2 2 
1 
4 

4_ 


Total Classrooms 


1 


17 


1 


1 


3 12 


Percent of class- 
rooms 


6 


94% 


6% 


6;^ 


185S m 



Note . From Stalllngs, J. A., & Kaskowltz, D. H. Follow Through 
classroom observation evaluation 1972-1973 (SRI Project URU-7370). 
Menlo Park, California: Stanford Research Institute, August 1974. 
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Table 3 



TEACHER WITH SMALL GROUP (Variable 88)— UNIVERSITY OF KANSAS 



First Grade Classrooms Third Grade Classrooms 
with Implementation with Implementation 
Scores of Scores of 

Sites 1234512345 

NYC P. S. 77X 11 2 

Philadelphis VI, Pa. 3 1 112 

Portageville. Mo. 4 12 

Kansas City, Mo. 13 4 

Louisville, 4 ' 4 

Total classrooms 5 13 1 1 1 14 



Percent of class- 
rooms 28% 72% 6% 6% 6% 82% 



Note . From Stallings, J. A., & Kaskowltz, D. H. Follow Through 
classroom observation evaluation 1972-1973 {SRI Project URU-7370). 
Menlo Park, California: Stanford Research Institute, August 1974. 
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Table 4 

READING, ALPHABET, LANGUAGE DEVELOPMENT (Variable 67)-- 
UNIVERSITY DF KANSAS 



Sites 



First Grade Classrooms 
with Implementation 
Scores of 

1 2 3 4 5 



Third Grade Classrooms 
with Implementation 
Scores of 

1 2 3 4 5 



NYC P.S. 77X 1 
Philadelphia, VI, Pa. 1 
Portageville, Mo, 
Kansas City, Mo. 1 
Louisville, Ky. 



Total Classrooms 1 2 1 



1 
2 
4 
1 
4 



1 
1 
2 
1 



1 

2 

2 
4 



12 



Percent of class- 
rooms 6% 11% 63S 11% 67% 



6% 12% 29% 53% 



Note . From Stallings, J. A., & Kaskowitz, D. H. Follow Through 
classroom observation evaluation 1972-1973 (SRI Project URU-737D). 



Menlo Park, California: Stanford Research Institute, August 1974. 
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Table 5 

ADULT COMMUNICATION OR ATTENTION FOCUS» ONE CHILD 
(Variable 438a)--UNIVERSITY OF KANSAS 



Sites 



First Grade Classrooms 
with Implementation 
Scores of 

1 2 3 4 5 



Third Grade Classrooms 
with Implementation 
Scores of 

1 2 3 4 5 



NYC P.S. 77X 
Philadelphia VI. Pa. 
Portageville» Mo. 
Kansas City» Mo. 
Louisville» Ky, 

Total classrooms 



1 

2 



1 
2 
4 
1 
4 



12 



2 
2 
3 
4 
4 



1 15 



Percent of class- 
rooms 11% 22% 67% 



6% 



6% BB% 



Note. From Stallingsi J. A., & Kaskowitzi, D. H. Follow Through 
classroom observation evaluation 1972-1973 (SRI Project URU-7370). 
Menlo Park» California: Stanford Research Institute* August 1974. 
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Figure Captions 

Figure K The first set of criteria that must be met by a 
teacher or aide to qualify for Behavior Analysis certification 
are the Instructional Teaching Criteria, 

Figure 2. The second set of performance criteria^ the 
Exchange Teaching Criteria^ are used to evaluate the operation 
of the classroom motivation system. 

Figure 3» When training consisting of setting a criterion^ 
modeling by the trainer^ and graphic and verbal feedback were 
employed^ the rate of contacts made to on-task children improved. 
The results were replicated with six trainees* 

Figure 4. Setting a criterion^ modeling^ and feedback were 
used together to improve the use of descriptive praise with tokens. 
The data show that following training^ about 7B% of all tokens 
were accompanied by a verbal statement clearly stating to the 
child what the token was being given for. 

Figure 5* A preliminary analyses of the effects of the 
certification program are shown* Children in classrooms whose 
teachers were certified completed more work than children in 
classrooms with teachers that did not meet the certification 
criteria. The performance of children taught by certified teachers 
maintained after certification had been granted. 
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Figure 6. First grade children with teachers who had met all 
of the B*A* certification criteria performed an average of four 
months higher on a year-end achievement test than children in the 
same site and grade level* but with teachers who did not meqt all 
of the criteria. The scores of 74 percent of the children with 
certified teachers were at or above the expected grade level whereas 
69 percent of the children with noncertified teachers scored at 
the expected level* 

Figure 7* The average score on a year-end test of reading 
was 3*3 for second grade children instructed by teachers meeting 
all of the B*A* criteria* and 2*5 for children instructed by teachers 
not meeting all of the criteria* a difference of eight months. 
Sixty-seven percent of the children with certified teachers scored 
at or above the expected grade level on the test* whereas only 40 
percent of the children with noncertified teachers scored at least 
at grade level . 

Figure 8* In grade three* the average achievement test score 
was 4*2 for children with certified teachers and 3.3 for children 
with teachers not meeting all of the criteria* a difference of nine 
months. Forty-six percent of the children with certified teachers 
were at or above grade level* whereas only 36 percent of the child- 
ren with teachers not meeting all of the criteria were at least 
at grade level on the test. 
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Figure 9, The mean rating given to the B,A, training program 
by teachers and aides was 6.0 in Spring, 1975. 

Figure 10, The bar graphs show the satisfaction ratings given 
to the sponsor by local project personnel in all of the B,A. sites. 

Figure 11, The bar graphs indicate the acceptability of the 
B,A, classroom methods as rated by school district personnel in all 
of the B,A, sites. 
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Figure 1 



Instructional Teaching Criteria 

These criteria must be met based on at least two 10-minute 
observations separated by at least four weeks: 

K 8035 of the children are on-task, 

2, 100% of your contacts are to children who are on-task, 

3, 100% of your contacts contain praise, 

4, 100% tokens given out are paired with praise. (Does not 
apply to classrooms using contingency contracting,) 

5, 90% of your contacts that include prompts also contain 
descriptive praise and tokens. (Tokens do not apply to 
classrooms using contingency contracting.) 

6, None of your contacts are disapprovals, 

7, Time-out procedures^ if needed^ are used appropriately, 
(See Don BushelU Jr,> Classroom Behavior > pp, 75-78.) 

8, Four children in your groups picked at random^ are work- 
ing at 80% accuracy, (The staff trainer will check the 
work of four students,) 
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Exchange Teaching Criteria 

These criteria must also be met on two observations separated 
by at least four weeks: 

1, Back-ups are prepared and ready prior to the start of the 
Instructional period. 

2. Prices and content of the back-ups vary at two exchanges 
observed on the same day. 

3. Each Instructional group sets own prices. 

4, Prices are the same for all children within each group* 
5* Children are free to choose any activity for which they 

have enough tokens. (Does not apply to classes using 

contingency contracting.) 
6. Children with too few tokens sit quietly during the exchange. 

(In contracting classrooms, children who do not complete 

their contracts sit quietly.) 
7* The first child In a group who Is ready to exchange Is 

allowed to. do so, without waiting for the other children 

to get ready. 

8. At least one back-up contributes to a relevant acadenic skill, 
g. Adults participate In the back-up activities. 

10. Teachers give praise for appropriate play during exchanges. 

11. At the end of the exchange, instruction begins with the 
first child who comes to the table. 
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Figure 3 
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Figure 6 
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Figure 7 
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Figure 8 
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Figure 9 

MEAN RATING BY TEACHERS AND AIDES 
ON 7-POINT SCALE 
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Figure 10 



SATISFACTION WITH SPONSOR'S SERVICES 
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Figure 11 

ACCEPTABILITY OF BEHAVIOR ANALYSIS 
CLASSROOM METHODS 
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